Supplementary Materials: Accelerated Stochastic Gradient Descent for Minimizing Finite Sums

1 Proof of the Proposition 1 We now prove the Proposition 1 that gives the condition of compactness of sublevel set. Proof. Let B(r) and S(r) denote the ball and sphere of radius r, centered at the origin. By affine transformation, we can assume that X∗ contains the origin O, X∗ ⊂ B(1), and X∗ ∩ S(1) = φ. Then, we have that for ∀x ∈ S(1), (∇f(x), x) ≥ f(x)− f(O) > 0, where we use convexity for the first inequality and O ∈ X∗ ∧ x / ∈ X∗ for the second inequality. We denote the minimum value of (∇f(x), x) on S(1) by α. Since (∇f(x), x) is positive continuous, we have α > 0. For ∀r ≥ 1 and ∀x ∈ S(r), we set x̂ = x/r ∈ S(1), then it follows that f(x) ≥ f(x̂) + (∇f(x̂), x− x̂) ≥ f(x̂) + (r − 1)(∇f(x̂), x̂) ≥ f∗ + (r − 1)α This inequality implies that if r > 1 + c−f∗ α , then we have f(x) > c for ∀x ∈ S(r). Therefore, sublevel set {x ∈ R; f(x) ≤ c} is a closed bounded set. 2 Proof of the Lemma 1 To prove Lemma 1, the following lemma is required, which is also shown in [1]. Lemma A. Let {ξi}i=1 be a set of vectors in R and μ denote an average of {ξi}i=1. Let I denote a uniform random variable representing a size b subset of {1, 2, . . . , n}. Then, it follows that, EI ∥ ∥ ∥ ∥ ∥ 1 b ∑